Great Schools RD with Demographic Characteristics - Preliminary Results

Magdalena Bennett

February 15, 2022

Context

Greatschools is a nonprofit organization which has as an objective to provide parents, schools, and communities with high-quality information. Currently, they provide both summary ratings and sub-ratings for schools in all states in the US. Ratings are shown in a 1 to 10 scale, with only integer values.

The objective of this study is to analyze the underlying continuous variable that is used to build these ratings, both for summary ratings as well as sub-ratings in different areas.

Summary Ratings

Summary ratings are built as a weighted average of sub-ratings for each school. For example:

\[CR_{its} = \omega^{T}_{its}\cdot TestRating_{its} + \omega^{G}_{its} GrowthRating_{its} + \omega^{E}_{its} EquityRating_{its} + \omega^{C}_{its} CollegeRating_{its}\] where \(CR_{its}\) is the continuous variable used to build the summary rating for school \(i\) in time \(t\) and state \(s\), which is the weighted average of four different ratings: Test scores, Growth or Student Progress, Equity, and College Readiness. Each \(\omega\) represents a weight for each school and each rating1. Finally, the summary rating \(R_{its}\) is built from the rounding of the continuous variable:

\[\lfloor CR_{its} \rceil = R_{its}\]

Preliminary Results

Effect of ratings on Black students

Using 2017 ratings2, we estimate the regression discontinuity for the total number of Black students for the years 2018 and 2019.

RD results for number of Black students in 2018 and 2019
N Black students 2018 N Black students 2019
1-2 19.844 22.552
(31.971) (32.905)
2-3 61.668** 58.778**
(24.486) (24.561)
3-4 -4.863 4.603
(14.09) (14.267)
4-5 -0.538 0.523
(12.531) (11.887)
5-6 -6.473 -7.22
(8.783) (8.302)
6-7 -23.714*** -22.416***
(6.238) (6.134)
7-8 -18.055** -16.357**
(8.302) (7.885)
8-9 -5.166 -5.718
(7.753) (7.62)
9-10 30.452 49.288
(24.014) (35.59)

We can also plot the RDs for the entire distribution of continuous ratings, where the cutoffs are for the midpoint between each integer rating (e.g. 2.5 is the cutoff between ratings 2 and 3). Below we show the plots for both 2018 and 2019 (Note: The plots are interactive, so you can zoom in to observe the discontinuities).



  1. These weights and ratings are provided directly by Greatschools. For more information of how these are built, see the ↩︎

  2. Ratings were rolled out late 2017 (e.g. November, December) and only a couple of states had ratings rolled out the first semester of 2019.↩︎